Exploration of milestone inscriptions in the EDH dataset
Petra Hermankova
01/10/2020
This notebook explores all the milestones in the Epigraphic Database Heidelberg (https://edh-www.adw.uni-heidelberg.de/, henceforth EDH). The primary aim is to:
- find all inscriptions categorized as ‘milestones’
- provide basic descriptive statistics: how many there are, what categories of monuments we have, what materials they are made from, where do they come from etc.
- display all the milestones on the map along with the map of Roman provinces and Roman roads
The script takes the reader through all the steps of selection, descriptive statistics and through the various displays on the interactive map. Address all the questions and comments to petra.hermankova@cas.au.dk.
1 Initial setup
1.1 Setup of the environment:
devtools::install_github("sdam-au/sdam") # loading SDAM custom package, if not working try devtools::install_github("mplex/cedhar", subdir="pkg/sdam")
#devtools::install_github("mplex/cedhar", subdir="pkg/sdam")
library(tidyverse)
library(sdam)
library(jsonlite)
library(getPass)
library(formatR)
library(leaflet)1.2 Loading data
- Load the dataset, if you have Sciencedata.dk credentials
resp = request("EDH_attrs_cleaned_2020-09-30.json", path="/sharingin/648597@au.dk/SDAM_root/SDAM_data/EDH/public", method="GET", cred=mycred_secret)- Make a list and tibble from the request function
list_json <- jsonlite::fromJSON(resp)
EDH_tibble = as_tibble(list_json)1.3 Exploring the data
Display the first six records of the dataset
head(EDH_tibble)2 Finding all the milestones
What attributes (columns) might contain milestone related information? Let’s display all the the attribute names.
EDH_tibble %>%
names()## [1] "responsible_individual" "type_of_inscription"
## [3] "letter_size" "not_after"
## [5] "literature" "work_status"
## [7] "height" "diplomatic_text"
## [9] "people" "depth"
## [11] "material" "type_of_monument"
## [13] "province_label" "width"
## [15] "transcription" "country"
## [17] "uri" "findspot_ancient"
## [19] "last_update" "modern_region"
## [21] "findspot_modern" "language"
## [23] "id" "edh_geography_uri"
## [25] "commentary" "trismegistos_uri"
## [27] "not_before" "findspot"
## [29] "year_of_find" "present_location"
## [31] "external_image_uris" "religion"
## [33] "fotos" "geography"
## [35] "social_economic_legal_history" "military"
## [37] "coordinates" "idno_tm"
## [39] "placenames_refs" "text_edition"
## [41] "origdate_text" "layout_execution"
## [43] "layout_execution_text" "support_objecttype"
## [45] "support_objecttype_text" "support_material"
## [47] "support_material_text" "support_decoration"
## [49] "keywords_term" "keywords_term_text"
## [51] "type_of_inscription_clean" "type_of_inscription_certainty"
## [53] "height_cm" "width_cm"
## [55] "depth_cm" "material_clean"
## [57] "type_of_monument_clean" "type_of_monument_certainty"
## [59] "province_label_clean" "province_label_certainty"
## [61] "country_clean" "country_certainty"
## [63] "findspot_ancient_clean" "findspot_ancient_certainty"
## [65] "modern_region_clean" "modern_region_certainty"
## [67] "findspot_modern_clean" "findspot_modern_certainty"
## [69] "findspot_clean" "findspot_certainty"
## [71] "origdate_text_clean"Based on the experience and previous explorations of the dataset, any milestone related information is most likely contained in the following attributes: type_of_inscription_clean, type_of_monumnet_clean, commentary,support_objecttype, support_objecttype_text, keywords_term and keywords_term_text.
2.1 Milestone as type of inscription
unique(EDH_tibble$type_of_inscription_clean)## [1] "epitaph" "honorific inscription"
## [3] "votive inscription" "defixio"
## [5] "owner/artist inscription" "mile-/leaguestone"
## [7] "acclamation" "boundary inscription"
## [9] "building/dedicatory inscription" "NULL"
## [11] "military diploma" "identification inscription"
## [13] "public legal inscription" "private legal inscription"
## [15] "label" "list"
## [17] "calendar" "seat inscription"
## [19] "elogium" "assignation inscription"
## [21] "prayer" "letter"
## [23] "adnuntiatio"2.1.1 How many inscriptions are categorised as a milestone?
milestone_insc <- EDH_tibble %>%
filter(type_of_inscription_clean == "mile-/leaguestone")
nrow(milestone_insc)## [1] 17302.2 Milestone as type of monument
unique(EDH_tibble$type_of_monument_clean)## [1] "tabula" "statue base"
## [3] "altar" "stele"
## [5] "tessera" "urn"
## [7] "block" "bar"
## [9] "mile-/leaguestone" "NULL"
## [11] "herm" "instrumentum domesticum"
## [13] "tile" "jewellery"
## [15] "cippus" "instrumentum militare"
## [17] "paving stone" "sarcophagus"
## [19] "base" "architectural member"
## [21] "cupa" "statue"
## [23] "slab" "instrumentum sacrum"
## [25] "grave monument" "sculpture"
## [27] "cliff" "relief"
## [29] "diptych" "bust"
## [31] "honorific/grave/votive column" "table"
## [33] "bench" "fountain"
## [35] "honorific/votive arch" "weapon"
## [37] "shield" "olla"
## [39] "fortification"2.2.1 How many inscriptions are categorised as milestone?
milestone_monument <- EDH_tibble %>%
filter(EDH_tibble$type_of_monument_clean == "mile-/leaguestone")
nrow(milestone_monument)## [1] 17352.3 Milestone in the commentary attribute
Inspect the commentary for milestone related comments
2.3.2 Explore the first six commentaries mentioning milestones
head(milestone_comment$commentary)## [1] " Meilenstein der via 19 des Itinerarium Antonini. Originalpublikation: Kaiser zwischen Tiberius und Vespasian. (B): Z.1 nicht gelesen."
## [2] " (A): ILER: Herkunft irrtümlich Caldas de Montbuy. (B): CIL, ILER: Z.2: AVG PA; Z.4 u. 5 noch vorhanden. Y: Meilenstein am ehesten Carus zuzuordnen, vielleicht auch Carinus oder Probus."
## [3] " Wiederverwendung des Meilensteins, ursprünglich vielleicht von Maximinus Thrax, da MAXIM auf der Rückseite lesbar. Z. 3 (Ende) u. Z. 6 (Ende): SEM bzw. NATO auf dem Foto nicht erkennbar. (B): Corradi Cervi: Z. 2: [vic]tori; AE 1983: Z. 2: victori; Giorgi: Z. 3: [triu]mfatori; Corradi Cervi: Z. 3: [triu]mphatori; Giorgi: Z. 4: [Aug]usto; Z. 5: [terrarum]; Corradi Cervi, AE 1983: Z. 5: [terr]arum; Giorgi: Z. 6: [n]ato."
## [4] " Y: Aufstellung des Meilensteins kurz vor dem Zug gegen die Goten wegen Fehlens von Gothicus Maximus. (A): Material: Marmor statt Kalkstein. Stein zum Fundort verschleppt."
## [5] " Teilweise Rasur eines früheren Textes auf diesem Meilenstein; die erhaltenen Reste ergeben keinen Sinn."
## [6] " 1. Meilenstein im Südosten von Lucania."2.4 Milestone in the support_objecttype attribute
The attribute support_object_type represents code for the Eagle Europeana standard vocabulary (https://www.eagle-network.eu/voc/objtyp/lod/89.html), where the Milestone has a LOD code 89.
How many there are?
milestone_lod_object<- EDH_tibble %>%
filter(support_objecttype =="89")
nrow(milestone_lod_object)## [1] 17302.5 Milestone in the support_objecttype_text attribute
Checking the consistency of support_objecttype and support_objecttype_text attributes to see if the attributes refer to the same type.
EDH_tibble %>%
select(support_objecttype, support_objecttype_text) %>%
dplyr::filter(str_detect(support_objecttype_text, "[M|m]eilen")) %>%
count(support_objecttype_text)The results are consistent and the free text corresponds with the LOD.
2.6 Milestone in the keywords_term attribute
The attribute keywords_term represents code for the Eagle Europeana standard vocabulary (https://www.eagle-network.eu/voc/typeins/lod/102.html), where the Milestone has a LOD code 102.
How many there are?
milestone_lod_insctype<- EDH_tibble %>%
filter(keywords_term =="102")
nrow(milestone_lod_insctype)## [1] 17252.7 Milestone in the support_objecttype_text attribute
Checking the consistency of keywords_term and keywords_term_text attributes to see if the attributes refer to the same type.
EDH_tibble %>%
select(keywords_term, keywords_term_text) %>%
dplyr::filter(str_detect(keywords_term_text, "[M|m]eilen")) %>%
count(keywords_term_text)The results are consistent and the free text corresponds with the LOD.
2.8 Combining all milestones
Now we will combine all milestones that have a word meilen- in commentary and are defined as milestones in the following attributes: type_of_inscription_clean, type_of_monumnet_clean,support_objecttype, keywords_term.
milestone_all <- EDH_tibble %>%
dplyr::filter(str_detect(commentary, "[M|m]eilen") | type_of_monument_clean == "mile-/leaguestone" | type_of_inscription_clean == "mile-/leaguestone" | support_objecttype =="89" | keywords_term =="102") 3 Descriptive overview
3.1 How many milestones total are in the EDH dataset when we combine all the search criteria?
nrow(milestone_all)## [1] 17703.2 How many % of the total number of EDH inscriptions represent milestones
nrow(milestone_all)/(nrow(EDH_tibble)/100)## [1] 2.1724193.3 Type of inscription
What different types of inscriptions belong to the discussed group of milestones?
milestone_all %>%
count(milestone_all$type_of_inscription_clean, sort = TRUE)3.4 Type of monument
What different types of monuments belong to the discussed group of milestones?
milestone_all %>%
count(milestone_all$type_of_monument_clean, sort = TRUE)3.5 Language
What language were written the inscriptions belonging to the discussed group of milestones?
unnest_auto(milestone_all, language) %>%
count(language, sort=TRUE)## Using `unnest_longer(language)`; no element has names3.6 Material
What different types of material is represented in the discussed group of milestones?
milestone_all %>%
count(material_clean, sort=TRUE)3.7 Dimensions
What are the dimensions of the discussed group of milestones?
milestone_dims <- milestone_all %>%
dplyr::select(height_cm, width_cm, depth_cm)
milestone_dims <- as.data.frame(milestone_dims)3.7.1 Basic statistical summary of Height in cm
An overview statistics for all Heights.
summary(milestone_dims$height_cm)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 5.7 72.0 119.5 124.3 170.0 330.0 4543.7.2 Basic statistical summary of Width in cm
An overview statistics for all Widths.
summary(milestone_dims$width_cm)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 6.00 35.00 43.00 45.11 50.00 279.00 5233.7.3 Basic statistical summary of Depth in cm
An overview statistics for all Depths.
summary(milestone_dims$depth_cm)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 0.10 17.73 25.00 26.25 35.00 63.00 16224 Spatial exploration
4.1 Ancient findspot
What kind of information do we have about ancient findspot of a milestone? Let’s see first 20 records.
milestone_all$findspot_ancient_clean[1:20]## [1] "Bracara Augusta - Lucus Augusti" "Bracara Augusta - Lucus Augusti"
## [3] "Bracara Augusta" "NULL"
## [5] "NULL" "NULL"
## [7] "NULL" "NULL"
## [9] "Thibilis - Calama" "Sebatum"
## [11] "Aquae Flaviae" "Matrica"
## [13] "Matrica" "Matrica"
## [15] "Matrica" "Fanum Fortunae"
## [17] "Divodurum" "NULL"
## [19] "Libyssa" "NULL"How many different ancient findspots do we have, including “NULL”?
length(unique(milestone_all$findspot_ancient_clean))## [1] 4314.2 Distribution by Roman provinces
How many milestones were found in individual Roman provinces? Display from the province containing the most to least milestones.
milestone_all %>%
count(province_label_clean, sort =TRUE)Display all the provinces and milestones in a chart:
milestone_all %>%
count(province_label_clean, sort =TRUE) %>%
ggplot(aes(y=province_label_clean, x=n), size = n) +
geom_point(color="red") +
coord_fixed(ratio = 7/1) +
labs(x = "Number of milestones", y = "Roman province", title = "Number of milestones per Roman Province", subtitle = "n = 1770" ) +
theme_linedraw(base_size = 12)
4.3 Map of all milestones (with available coordinates)
# selecting milestones with coordinates
coords_milestone<- as.data.frame(cbind(id = milestone_all$id,
coordinates = milestone_all$coordinates,
findspot_ancient_clean = milestone_all$findspot_ancient_clean,
type_of_inscription_clean = milestone_all$type_of_inscription_clean,
type_of_monument_clean = milestone_all$type_of_monument_clean,
not_before = milestone_all$not_before,
not_after = milestone_all$not_after,
commentary = milestone_all$commentary))
# milestones with no coordinates
coords_milestone_empty<- coords_milestone %>%
dplyr::filter(coordinates == "list()")
# milestones with coordinates
coords_milestone_full<- coords_milestone %>%
dplyr::filter(coordinates != "list()")
lat_long_milestone<- coords_milestone_full %>%
separate(col = coordinates, into = c("longitude", "latitude"), sep = ",")## Warning: Expected 2 pieces. Missing pieces filled with `NA` in 102 rows [9, 33,
## 42, 66, 87, 101, 168, 248, 249, 316, 317, 366, 368, 387, 409, 459, 655, 657,
## 717, 723, ...].lat_long_milestone$latitude <- as.numeric(str_replace(lat_long_milestone$latitude, pattern = "\\)", replacement=""))
lat_long_milestone$longitude <- as.numeric(str_replace(lat_long_milestone$longitude, pattern = "c\\(", replacement=""))## Warning: NAs introduced by coercion4.3.1 How many milestones have spatial coordinates
nrow(lat_long_milestone)## [1] 17704.4 Milestones as dots
map_milestones_dot <-leaflet(width="100%") %>%
addProviderTiles("Stamen.Watercolor")%>% # Add CartoDB map tiles
addCircles(lng = lat_long_milestone$longitude,
lat = lat_long_milestone$latitude, radius = 10, fill = TRUE, color= , fillColor = lat_long_milestone$type_of_monument_clean,
popup = paste0("<b> InscriptionID: </b>", lat_long_milestone$id,
"<br><b> Ancient findspot: </b>", lat_long_milestone$findspot_ancient,
"<br><b> Type of inscription: </b>", lat_long_milestone$type_of_inscription_clean,
"<br><b> Type of monument: </b>", lat_long_milestone$type_of_monument_clean,
"<br><b> Not before (date): </b>", lat_long_milestone$not_before,
"<br><b> Not after (date): </b>", lat_long_milestone$not_after,
"<br><b> Commentary: </b>", lat_long_milestone$commentary),
) %>%
addLegend(position = "topright",
colors = c("Blue"),
labels = c("Milestones (n=1770)"), opacity = 1,
title = "Position of all milestones (EDH)"
)## Warning in validateCoords(lng, lat, funcName): Data contains 102 rows with
## either missing or invalid lat/lon values and will be ignoredmap_milestones_dot4.5 Clustering of milestones
map_milestones_cluster <-leaflet(width="100%") %>%
#addProviderTiles("Esri.WorldTopoMap", group = "Topo") %>%
addProviderTiles("Esri.WorldImagery", group = "ESRI Aerial") %>%
addMarkers(lng = lat_long_milestone$longitude,
lat = lat_long_milestone$latitude,
popup = paste0("<b> InscriptionID: </b>", lat_long_milestone$id,
"<br><b> Ancient findspot: </b>", lat_long_milestone$findspot_ancient,
"<br><b> Type of inscription: </b>", lat_long_milestone$type_of_inscription_clean,
"<br><b> Type of monument: </b>", lat_long_milestone$type_of_monument_clean,
"<br><b> Not before (date): </b>", lat_long_milestone$not_before,
"<br><b> Not after (date): </b>", lat_long_milestone$not_after,
"<br><b> Commentary: </b>", lat_long_milestone$commentary),
clusterOptions = markerClusterOptions()
) %>%
addLegend(position = "topright",
colors = c("Blue", "Green", "Yellow", "Orange"),
labels = c("Individual inscription", "Small cluster", "Medium cluster", "Large cluster"), opacity = 1,
title = "Clustering of all milestones in the EDH dataset"
)## Warning in validateCoords(lng, lat, funcName): Data contains 102 rows with
## either missing or invalid lat/lon values and will be ignoredmap_milestones_cluster4.6 Milestones and Roman provinces (200 AD)
## Loading required package: sp## rgdal: version: 1.5-16, (SVN revision 1050)
## Geospatial Data Abstraction Library extensions to R successfully loaded
## Loaded GDAL runtime: GDAL 2.2.3, released 2017/11/20
## Path to GDAL shared files: /usr/share/gdal/2.2
## GDAL binary built with GEOS: TRUE
## Loaded PROJ runtime: Rel. 4.9.3, 15 August 2016, [PJ_VERSION: 493]
## Path to PROJ shared files: (autodetected)
## WARNING: no proj_defs.dat in PROJ.4 shared files
## Linking to sp version:1.4-2##
## Attaching package: 'raster'## The following object is masked from 'package:dplyr':
##
## select## The following object is masked from 'package:tidyr':
##
## extract## Linking to GEOS 3.6.2, GDAL 2.2.3, PROJ 4.9.3source Roads: http://awmc.unc.edu/awmc/map_data/shapefiles/ba_roads/ source Roman provinces (AD 117): http://awmc.unc.edu/awmc/map_data/shapefiles/cultural_data/political_shading/roman_empire_ad_117/shape/
## Warning in dir.create("../data/ba_roads"): '../data/ba_roads' already exists## Warning in unzip("../data/ba_roads.zip", exdir = "../data/ba_roads/"): error 1
## in extracting from zip file## Warning in dir.create("../data/roman_empire_200_ad_provinces"): '../data/
## roman_empire_200_ad_provinces' already exists## Warning in unzip("../data/roman_empire_ad_200_provinces.zip", exdir = "../data/
## roman_empire_200_ad_provinces"): error 1 in extracting from zip file## Reading layer `ba_roads' from data source `/home/petra/Github/epigraphic_roads/data/ba_roads/ba_roads.shp' using driver `ESRI Shapefile'
## replacing null geometries with empty geometries
## Simple feature collection with 2739 features and 26 fields (with 1 geometry empty)
## geometry type: LINESTRING
## dimension: XY
## bbox: xmin: -9.29586 ymin: 23.90006 xmax: 52.88473 ymax: 56.52055
## CRS: 4326## Reading layer `roman_empire_ad_200_provinces' from data source `/home/petra/Github/epigraphic_roads/data/roman_empire_200_ad_provinces/roman_empire_ad_200_provinces.shp' using driver `ESRI Shapefile'
## replacing null geometries with empty geometries
## Simple feature collection with 81 features and 11 fields (with 1 geometry empty)
## geometry type: LINESTRING
## dimension: XY
## bbox: xmin: NA ymin: NA xmax: NA ymax: NA
## CRS: 4326map_milestones_provinces<- leaflet(width="100%") %>%
#addProviderTiles("Esri.WorldTopoMap", group = "Topo") %>%
#addProviderTiles("Esri.WorldImagery", group = "ESRI Aerial") %>%
addProviderTiles("Stamen.Watercolor")%>% # Add CartoDB map tiles
addPolylines(data = provinces200, color = "red", weight = 2, opacity = 0.7) %>%
addCircles(lng = lat_long_milestone$longitude,
lat = lat_long_milestone$latitude, radius = 10, fill = TRUE, color = "blue" , fillColor = "red",
popup = paste0("<b> InscriptionID: </b>", lat_long_milestone$id,
"<br><b> Ancient findspot: </b>", lat_long_milestone$findspot_ancient,
"<br><b> Type of inscription: </b>", lat_long_milestone$type_of_inscription_clean,
"<br><b> Type of monument: </b>", lat_long_milestone$type_of_monument_clean,
"<br><b> Not before (date): </b>", lat_long_milestone$not_before,
"<br><b> Not after (date): </b>", lat_long_milestone$not_after,
"<br><b> Commentary: </b>", lat_long_milestone$commentary),
) %>%
addLegend(position = "topright",
colors = c("Blue", "Red"),
labels = c("Milestones", "Roman Provinces (200 AD)"), opacity = 1,
title = "Milestones in the Roman Empire"
)## Warning in validateCoords(lng, lat, funcName): Data contains 102 rows with
## either missing or invalid lat/lon values and will be ignoredmap_milestones_provinces4.7 Milestones and Roman roads
map_milestones_roads<- leaflet(width="100%") %>%
addProviderTiles("Stamen.Watercolor")%>% # Add CartoDB map tiles
addProviderTiles("Stamen.TerrainBackground")%>% # Add CartoDB map tiles
#addProviderTiles("Esri.WorldTopoMap", group = "Topo") %>%
#addProviderTiles("Esri.WorldImagery", group = "ESRI Aerial") %>%
setView( lng = 22.326743, lat = 46.897122, zoom = 4 ) %>%
#setMaxBounds(lat1=43.633977, lng1 =-11.227926 , lat2=35.133882 , lng2=50.882336) %>%
addPolylines(data = roads, color = "purple", weight = 1, opacity = 0.7) %>%
addCircles(lng = lat_long_milestone$longitude,
lat = lat_long_milestone$latitude, opacity = 0.5, radius = 10, fill = TRUE, color = "blue" , fillColor = "red",
popup = paste0("<b> InscriptionID: </b>", lat_long_milestone$id,
"<br><b> Ancient findspot: </b>", lat_long_milestone$findspot_ancient,
"<br><b> Type of inscription: </b>", lat_long_milestone$type_of_inscription_clean,
"<br><b> Type of monument: </b>", lat_long_milestone$type_of_monument_clean,
"<br><b> Not before (date): </b>", lat_long_milestone$not_before,
"<br><b> Not after (date): </b>", lat_long_milestone$not_after,
"<br><b> Commentary: </b>", lat_long_milestone$commentary),
) %>%
addLegend(position = "topright",
colors = c("Blue", "Purple"),
labels = c("Milestones", "Roman roads (BA atlas)"), opacity = 1,
title = "Milestones and Roman roads"
)## Warning in validateCoords(lng, lat, funcName): Data contains 102 rows with
## either missing or invalid lat/lon values and will be ignoredmap_milestones_roads4.8 Location of milestones compared with all inscriptions
# inscriptions with coordinates
lat_long_EDH <- EDH_tibble %>%
separate(col = coordinates, into = c("longitude", "latitude"), sep = ",")## Warning: Expected 2 pieces. Missing pieces filled with `NA` in 2464 rows [20,
## 34, 64, 71, 94, 149, 212, 221, 296, 311, 356, 382, 388, 390, 405, 406, 409, 412,
## 418, 421, ...].lat_long_EDH